Methods, systems, and media for controlling audio output

ABSTRACT

Methods, systems, and media for controlling audio output are provided. In some implementations, a method for controlling audio output is provided, the method comprising: receiving, by a computing device, audio content to be presented; determining that an output port of the computing device is connected to a corresponding connector; providing the audio content at the output port; recording a first portion of audio using a microphone; determining that the audio content is present in the first portion of audio; continuing to provide the audio content at the output port; recording a second portion of audio using the microphone; determining that the audio content is not present in the second portion of audio; and presenting the audio content using a speaker of the computing device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Pat. Application No.17/019,579, filed Sep. 14, 2020, which is a continuation of U.S. Pat.Application No. 16/390,508, filed Apr. 22, 2019, which is a continuationof U.S. Pat. Application No. 15/923,386, filed Mar. 16, 2018, which is acontinuation of U.S. Pat. Application No. 15/164,406, filed May 25,2016, each of which is hereby incorporated by reference herein in itsentirety.

TECHNICAL FIELD

Methods, systems and media for controlling audio output are provided.

BACKGROUND

Devices that interact with users primarily through audio commands givenby the user, and audio feedback provided to the user through a built-inspeaker have recently become more popular. With such devices, users canask questions, dictate messages, listen to written messages that areread out loud by the device, play music, and perform other tasks throughaudio interaction without having to interact with a graphical userinterface. Some users may wish to connect such devices to externalspeakers, but this may limit the usefulness of the device if theexternal speakers are off, or otherwise not playing the audio output bythe device.

Accordingly, new methods, systems and media for controlling audio outputare provided.

SUMMARY

In accordance with some implementations of the disclosed subject matter,mechanisms for controlling audio output are provided.

In accordance with some implementations of the disclosed subject matter,a method for controlling audio output is provided, the methodcomprising: receiving, by a computing device, audio content to bepresented; determining that an output port of the computing device isconnected to a corresponding connector; in response to determining thatthe output port is connected to the corresponding connector, providingthe audio content at the output port; recording a first portion of audiousing a microphone of the computing device; determining that the audiocontent is present in the first portion of audio; subsequent todetermining that the audio content is present in the first portion ofaudio, continuing to provide the audio content at the output port;recording a second portion of audio using the microphone; determiningthat the audio content is not present in the second portion of audio;and in response to determining that the audio content is not present inthe second portion of audio, presenting the audio content using aspeaker of the computing device.

In accordance with some implementations of the disclosed subject matter,a system for controlling audio output is provided, the systemcomprising: a computing device comprising: an output port; a microphone;a speaker; and a hardware processor that is programmed to: receive audiocontent to be presented; determine that the output port of the computingdevice is connected to a corresponding connector; in response todetermining that the output port is connected to the correspondingconnector, provide the audio content at the output port; record a firstportion of audio using the microphone; determine that the audio contentis present in the first portion of audio; subsequent to determining thatthe audio content is present in the first portion of audio, continue toprovide the audio content at the output port; record a second portion ofaudio using the microphone; determine that the audio content is notpresent in the second portion of audio; and in response to determiningthat the audio content is not present in the second portion of audio,present the audio content using the speaker.

In accordance with some implementations of the disclosed subject matter,a non-transitory computer-readable medium containing computer executableinstructions that, when executed by a processor, cause the processor toperform a method for controlling audio output is provided, the methodcomprising: receiving, by a computing device, audio content to bepresented; determining that an output port of the computing device isconnected to a corresponding connector; in response to determining thatthe output port is connected to the corresponding connector, providingthe audio content at the output port; recording a first portion of audiousing a microphone of the computing device; determining that the audiocontent is present in the first portion of audio; subsequent todetermining that the audio content is present in the first portion ofaudio, continuing to provide the audio content at the output port;recording a second portion of audio using the microphone; determiningthat the audio content is not present in the second portion of audio;and in response to determining that the audio content is not present inthe second portion of audio, presenting the audio content using aspeaker of the computing device.

In accordance with some implementations of the disclosed subject matter,a system for controlling audio output is provided, the systemcomprising: means for outputting an audio signal; means for recordingaudio; means for presenting audio content; means for receiving audiocontent to be presented; means for determining that the means foroutputting the audio signal is connected to a corresponding connector;means, responsive to the means for determining that the output port isconnected to the corresponding connector, for providing the audiocontent at the means for outputting the audio signal; means forrecording a first portion of audio using means for recording audio;means for determining that the audio content is present in the firstportion of audio; means for continuing to provide the audio content atthe means for outputting the audio signal; means for recording a secondportion of audio using the means for recording audio; means fordetermining that the audio content is not present in the second portionof audio; and means, responsive to the means for determining that theaudio content is not present in the second portion of audio, forpresenting the audio content using a speaker of the computing device.

In some implementations, the system further comprises: means fordetermining that the means for outputting the audio signal is no longerconnected to the corresponding connector; and means, responsive to themeans for determining that the means for outputting audio is no longerconnected to the corresponding connector, for inhibiting the audiocontent from being provided to the means for outputting the audiosignal.

In some implementations, the means for determining that the audiocontent is present in the first portion of audio comprises: means forgenerating an audio fingerprint based on the first portion of audio;means for determining that the audio fingerprint based on the firstportion of audio matches an audio fingerprint generated based on theaudio content; and means, responsive to the means for determining thatthe audio fingerprint based on the first portion of audio matches anaudio fingerprint generated based on the audio content, for determiningthat the audio content is present in the first portion of audio.

In some implementations the means for determining that the audio contentis present in the first portion of audio comprises: means forcalculating a difference between a portion of the audio content providedat the output port and the first portion of audio; and means fordetermining that the audio content is present in the first portion ofaudio based on the difference.

In some implementations, the means for outputting the audio signal is a3.5 millimeter jack.

In some implementations, the system further comprises means, responsiveto the means for determining that the audio content is present in thefirst portion of audio, for inhibiting the audio content from beingpresented using the speaker of the computing device.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects, features, and advantages of the disclosed subjectmatter can be more fully appreciated with reference to the followingdetailed description of the disclosed subject matter when considered inconnection with the following drawings, in which like reference numeralsidentify like elements.

FIG. 1 shows an example of a system for controlling audio output inaccordance with some implementations of the disclosed subject matter.

FIG. 2 shows an example of a generalized schematic diagram of a systemon which the mechanisms for controlling audio output as described hereincan be implemented in accordance with some implementations of thedisclosed subject matter.

FIG. 3 shows an example of hardware that can be used to implement one ofthe computing devices, the external sound system and the contentdelivery server depicted in FIG. 2 in accordance with someimplementations of the disclosed subject matter.

FIG. 4 shows an example of a process for controlling audio output inaccordance with some implementations of the disclosed subject matter.

DETAILED DESCRIPTION

In accordance with some implementations of the disclosed subject matter,mechanisms (which can include methods, systems and media) forcontrolling audio output are provided.

In some implementations, the mechanisms described herein can control theoutput of audio from a computing device that can interact with a userthrough an audio user interface to one or more external speakers. Insome implementations, a user can request that the computing devicepresent audio content using any suitable technique or combination oftechniques. For example, the user can make the request through a voicecommand by speaking one or more words, through an application on anotherdevice such as a smartphone or tablet computer, through a graphical userinterface of the computing device, and/or through any other suitableuser interface. In some implementations the audio content can be anysuitable audio content and/or can be provided from any suitable source.For example, the audio content can be content that is stored as one ormore audio signals such as music or an audio book. As another example,the audio content can be content that is converted to audio from textsuch as an article, a message (e.g., an email, a text message, etc.), arecipe, a web page, etc. In such examples, the audio content can beavailable from any suitable source such as internal memory of thecomputing device, internal memory of another device local device (e.g.,a smartphone, a tablet computer, a personal computer, network attachedstorage, etc.) from which the computing device can access the content,remote storage (e.g., a server) that is associated with a user of thecomputing device, a service that provides access to audio content (e.g.,a streaming audio service, a streaming video service, etc.), a webserver that provides access to web pages, and/or any other suitablesource. In a more particular example, a user can speak one or more wordsrequesting that the computing device play music from a particular artistfrom a particular streaming music service which the user may accessthrough a user account.

In some implementations, the computing device can receive the audiocontent to be presented and determine whether to present the audiocontent through an internal speaker or to output the audio content forpresentation by an external speaker. For example, a user can connect anaudio output of the computing device to an external sound system thatincludes external speakers that may provide a subjectively moreenjoyable audio experience for the user. In a more particular example,the user can connect a cable having a 3.5 millimeter (mm) plug at oneend to a 3.5 millimeter output jack of the computing device, and canconnect the other end of the cable to a jack of an audio video receiverthat is connected to one or more external speakers.

In some implementations, the computing device can detect that the audiooutput is connected to a corresponding connector and attempt todetermine whether audio that is provided to the connector is presentedby external speakers. For example, the computing device can detect thatthe 3.5 mm jack is connected to something that has one or moreproperties that a 3.5 mm plug has. In some implementations, when theconnector is detected, the computing device can provide the audiocontent to the audio output, and record a signal produced by amicrophone of the computing device. In some implementations, thecomputing device can compare the recorded signal to the output signal todetermine whether the signal that is being output is being played by anynearby (e.g., close enough to be recorded at sufficient volume) externalspeakers. For example, the computing device can record its surroundingswith a microphone and compare the recorded signal to the music beingoutput to the audio output to determine whether that same music has beenrecorded by the microphone.

In some implementations, if the audio that is recorded does not matchthe audio that is being provided to the audio output, the computingdevice can present the audio content using the internal speaker. Forexample, if the audio video receiver is powered down, the computingdevice can determine that the music being provided to the 3.5 mm jack isnot present in the recorded audio signal, and can begin playing themusic over the internal speaker.

Turning to FIG. 1 , an example 100 of a system for controlling audiooutput is shown in accordance with some implementations of the disclosedsubject matter. In some implementations, system 100 can include acomputing device 102, which can include an output connector 104. Outputport 104 can be any suitable output connector for communicating audioinformation with an external sound system 106. For example, output port104 can be a 3.5 mm jack that accepts a 3.5 mm connector, a fiber opticaudio port (e.g., that accepts a TOSLINK connector), an RCA jack, aHigh-Definition Multimedia Interface (HDMI) port, a Universal Serial Bus(USB) port (e.g., a USB 2.x port, a USB 3.x port, etc.), a THUNDERBOLTport, an Ethernet port (whether wired or wireless), any other suitableport and/or any suitable combination of ports.

In some implementations, external sound system 106 can be any suitabledevice that can receive an audio signal and output the audio signal(potentially through one or more intermediate devices) to one or morespeakers. For example, external sound system 106 can be an audio video(AV) receiver, a speaker, an amplifier, an audio switch, an HDMI switch,any other suitable external sound system and/or any suitable combinationthereof. More particularly, external sound system 106 can include,speakers, hardware and/or software for rendering media content such asanalog and/or digital audio and/or video data, a power supply, etc.External sound system 106 can include various input ports for receivingaudio and/or video data from various sources. Such input ports caninclude one or more HDMI ports, one or more component video ports, oneor more composite video ports, one or more USB ports, one or moreS-Video ports, one or more TOSLINK ports, one or more coaxial ports, oneor more Ethernet ports (whether wired or wireless), etc. Additionally oralternatively, in some implementations, external sound system 106 can beconfigured to receive audio over any suitable wireless connection, suchas over a connection to a local area network (e.g., via a connectionthat complies with one of the IEEE 802.11x family of protocols which aresometimes referred to as Wi-Fi connections), or a wireless connectionbetween external sound system 106 and computing device 102, such as anad hoc wireless network connection, a short-range wireless communicationprotocol (e.g., Bluetooth, Bluetooth Low Energy, Wireless USB, etc.),etc.

In some implementations, computing device 102 can be connected toexternal sound system 106 by a connection 110. Computing device 102 andexternal sound system 106 can be connected using any suitabletechnique(s). For example, connection 110 can include a suitable cable,such as a stereo cable, an RCA cable, an HDMI cable, that can beconnected between output port 104 and a port of external audio system106. As another example, connection 110 can include a wirelessconnection, such as a connection to a local area network (e.g., via aWi-Fi connection), or a wireless connection established betweencomputing device 102 and external sound system 106, such as an ad hocwireless network connection, a short-range wireless communicationprotocol (e.g., Bluetooth, Bluetooth Low Energy, Wireless USB, etc.),etc. Additionally or alternatively, media streaming device 102 can beincluded as hardware and/or software of external sound system 106.

In some implementations, computing device 102 can be coupled to a userdevice 112 using a communications link 114. User device 112 can be anysuitable device, such as a smartphone, a tablet computer, a wearablecomputer, a laptop computer, a personal computer, an electronic reader,a digital media receiver, a smart television, a game console, any othersuitable computing device, or any suitable combination thereof. In someimplementations, communications link 114 can include any suitablecommunications link, for example, as described below in connection withFIG. 2 . Additionally, communications link 114 can include a network,such as a local area network (LAN) having wired and/or wirelessconnections. Additionally or alternatively, communications link 114 caninclude a wireless connection between user device 112 and computingdevice 102, such as an ad hoc wireless network connection, a short-rangewireless communication protocol (e.g., Bluetooth, Bluetooth Low Energy,Wireless USB, etc.), etc. In some implementations, communicationsbetween user device 112 and computing device 102 over communicationslink 114 can be encrypted using any suitable encryption technique(s).For example, messages sent from user device 112 to computing device 102can be encrypted using any suitable encryption scheme, for example,using a public key-private key pair for encryption and decryption ofmessages. As another example, communications over communications link114 can be encrypted using Transport Layer Security (TLS).

In some implementations, user device 112 can include a display 116 forpresenting a user interface to a user. In some examples describedherein, user device 112 can be a mobile computing device such as asmartphone or a tablet computer, and display 116 can include atouchscreen for receiving input and displaying a user interface and/ormedia content to a user.

In some implementations, computing device 102 and/or user device 112 cancommunicate with a content discovery server 122 over a communicationslink 126. Computing device 102 and/or user device 112 can communicatewith content discovery server 122 to discover and/or browse content madeavailable by a content delivery server 120. For example, a user can useuser device 112 to find media content to be presented by interactingwith content discovery server 122. In some implementations, such mediacontent can be presented using display 116, one or more speakers, anaudio output (e.g., a 3.5 mm jack), any other suitable presentationdevice, or any suitable combination thereof. In some implementations,user device 112 can present a media content item that was selected usingcontent discovery server 122, by requesting the media content item fromcontent delivery server 120 and receiving the media content item over acommunications link 124. In some implementations, each of contentdelivery server 120 and/or content discovery server 122 can beimplemented on any suitable number of servers. Additionally oralternatively, in some implementations, the functions performed bycontent delivery server 120 and content discovery server 122 can beperformed by the same device (e.g., by a common server computer). Insome implementations, multiple content delivery servers can providecontent to user device 112 in response to a request for content. Forexample, in response to a request for a particular media content itemfrom user device 112, content delivery server 120 can cause anadvertisement to be provided to user device 112 prior to, during and/orafter the requested content item is presented. In such an example, theserver that provides the advertisement can be separate from the serverthat provides the requested media content item.

In some implementations, communications links 124 and/or 126 can includea local area network (e.g., a home network) and/or a non-local network(e.g., the Internet). For example, computing device 102 and/or userdevice 112 can be connected to a common LAN such as a home network, andcan connect to content discovery server 122 and/or content deliveryserver 120 through the Internet. As another example, content deliveryserver 120 and/or content discovery server 122 can be connected to a LANthat is common to computing device 102 and/or user device 112. In a moreparticular example, content delivery server 120 and/or content discoveryserver 122 can be located remotely from computing device 102 and/or userdevice 112, and communications links 124 and/or 126 can include anon-local network such as the Internet for accessing content fromcontent delivery server 120 and/or discovering content using contentdiscovery server 122. In another more particular example, contentdelivery server 120 and/or content discovery server 122 can be locatedlocally to computing device 102 and/or user device 112, andcommunications links 124 and/or 126 can include a local network, such asa home network, to which content delivery server 120 and/or contentdiscovery server 122 are connected. In such an example, a non-localnetwork of communications links 124 and/or 126 (if it exists) may not beused when accessing content from content delivery server 120 and/ordiscovering content using content discovery server 122.

In some implementations, a user of user device 112 can cause mediacontent made available by content delivery server 120 to be transmittedto computing device 102, which can, in turn, present the media contentusing a speaker integrated in computing device 102 and/or output themedia content to external sound system 106.

In some implementations, a user of computing device 102 can interactwith computing device 102 to request media content. For example,computing device 102 can receive record an audio signal and determinethat a user said one or more words, can interpret the words as aninstruction to request media content, and can take one or more actionsbased on the instruction.

In some implementations, when a media content item is to be presentedfrom content delivery server 120 using computing device 102, the mediacontent item can be transmitted to computing device 102 over acommunications link 128. Communications link 128 can include a localarea network (e.g., a home network) and/or a non-local network (e.g.,the Internet). For example, computing device 102 and user device 112 canboth be connected to a common LAN such as a home network, and canconnect to content delivery server 120 through the Internet. As anotherexample, content delivery server 120 can be connected to a LAN that iscommon to user device 112 and/or computing device 102. In a moreparticular example, content delivery server 120 can be located remotelyfrom computing device 102 and/or user device 112, and communicationslink 128 can include a non-local network such as the Internet foraccessing content from content delivery server 120. In another moreparticular example, content delivery server 120 can be located locallyto computing device 102 and/or user device 112, and communications link128 can include a local network, such as a home network, to which bothcomputing device 102 and content delivery server 120 are connected. Insuch an example, a non-local network of communications link 128 (if itexists) may not be used when accessing content from content deliveryserver 120.

In some implementations, a user can choose a media content item to bepresented using computing device 102 from a memory of user device 112(e.g., the user can choose a video that is stored locally on user device112). Additionally or alternatively, a user can choose a media contentitem to be presented using computing device 102 that is stored onanother computing device present on the same local area network (e.g.,network attached storage, a memory of a personal computer on a homenetwork, memory on a server on the same network, etc.). Such a mediacontent item stored on a computing device connected to a local areanetwork can be transmitted from user device 112 to computing device 102over communications link 114 (or any other suitable communicationslink), without being transmitted through content delivery server 120.

FIG. 2 shows an example 200 of a generalized schematic diagram of asystem on which the mechanisms for controlling audio output as describedherein can be implemented in accordance with some implementations of thedisclosed subject matter. As illustrated, system 200 can include one ormore user devices 112. User devices 112 can be local to each other orremote from each other. User devices 112 can be connected by one or morecommunications links 202 to a communications network 204 that can belinked via a communications link 210 to computing device(s) 102, via acommunications link 208 to content delivery server 120, viacommunications link 212 to content discovery server 122, and viacommunications links 202 to other user devices 112.

In some implementations, each of the user devices 112, content deliveryserver 120, content discovery server 122 and computing device 102 can beany of a general purpose device such as a computer or a special purposedevice such as a client, a server, etc. Any of these general or specialpurpose devices can include any suitable components such as a hardwareprocessor (which can be a microprocessor, digital signal processor, acontroller, a field-programmable gate array, a complex programmablelogic device, an application-specific integrated circuit, etc.), memory,communication interfaces, display controllers, input devices, etc. Forexample, user device 112 can be implemented as a smartphone, a tabletcomputer, a wearable computer, a personal computer, a laptop computer, agaming console, a digital media receiver, a set-top box, a smarttelevision, a server, etc. As another example, computing device 102 canbe implemented as a wireless speaker, a voice command device, a devicethat provides a digital assistant service, a personal computer, a laptopcomputer, a gaming console, a smart television, a smartphone, a tabletcomputer, a wearable computer, a digital media receiver, a set-top box,a smart television, a server, any suitable home automation device thatis connected to and/or can connect to an internal and/or externalspeaker, etc.

Communications network 204 can be any suitable computer network orcombination of such networks including the Internet, an intranet, awide-area network (WAN), a local-area network (LAN), a wireless network,a digital subscriber line (DSL) network, a frame relay network, anasynchronous transfer mode (ATM) network, a virtual private network(VPN), etc. Communications links 202, 208, 210 and 212 can be anycommunications links suitable for communicating data among user devices112, computing device 102, content delivery server 120, and contentdiscovery server 122, such as network links, dial-up links, wirelesslinks, hard-wired links, any other suitable communications links, or anysuitable combination of such links. User devices 112 can discover,browse, download, stream, store, playback, transmit, and/or otherwisepresent content stored locally at user device 112 and/or stored remotelyat content delivery server 120 using any suitable technique(s).Additionally, in some implementations, computing device 102 candiscover, browse, download, stream, store, playback, transmit, and/orotherwise present content stored locally at computing device 102 and/orstored remotely at content delivery server 120 using any suitabletechnique(s).

System 200 can include computing device 102 as described above inconnection with FIG. 1 , and can include an application installed onand/or executed by on one or more of external sound systems 106, whichcan replicate the functions of computing device 102 described herein.

System 200 can include one or more content delivery servers 120. Contentdelivery server 120 can be any suitable server for providing access tomedia content items in accordance with the mechanisms described herein,such as a processor, a computer, a data processing device, and/or anysuitable combination of such devices.

System 200 can include one or more content discovery servers 122.Content discovery server 122 can be any suitable server for facilitatingdiscovery and/or browsing of media content items accessible on contentdelivery server 120 in accordance with the mechanisms described herein,such as a processor, a computer, a data processing device, and/or anysuitable combination of such devices.

In some implementations, communications link 114 described above inconnection with FIG. 1 can include communications link 202,communication network 204, and communications link 210; communicationslink 126 described above in connection with FIG. 1 can includecommunications link 202 and/or 210, communication network 204, andcommunications link 212; and communications link 124 described above inconnection with FIG. 1 can include communications link 202 and/or 210,communication network 204, and communications link 208.

FIG. 3 shows an example 300 of hardware that can be used to implementone of computing devices 102, external sound system 106 and contentdelivery server 120 depicted in FIG. 2 in accordance with someimplementations of the disclosed subject matter. Referring to FIG. 3 ,computing device 102 can include a hardware processor 312, one or moremicrophones 314, memory 316, a transmitter/receiver 318, one or moreaudio output ports 320 and/or one or more speakers 321 which can beinterconnected. In some implementations, memory 316 can include astorage device (such as a non-transitory computer-readable medium) forstoring a computer program for controlling hardware processor 312.

Hardware processor 312 can use the computer program to present and/orrecord audio using microphone(s) 314 and/or to interact with a user(e.g., by capturing words spoken by the user using microphone(s) 314and/or by presenting audio using speaker(s) 321) to allow a user to,among other things, request media content to be presented usingcomputing device 102. It should also be noted that data received throughcommunications link 210, and/or any other communications links can bereceived from any suitable source. In some implementations, hardwareprocessor 312 can send and receive data through communications link 210,communications link 114, communications link 124, communications link126, connection 110, and/or any other communications links using, forexample, a transmitter, receiver, transmitter/receiver, transceiver, orany other suitable communication device, such as transmitter/receiver318. Microphone(s) 314 can include any suitable number and/or type ofmicrophones. Transmitter/receiver 318 can include any suitabletransmitter and/or receiver for transmitting and/or receiving, amongother things, media content including audio and/or video content. Audiooutput 320 can include any suitable audio port or audio ports (and/orvideo ports in some implementations) for outputting audio to an externalsound system (e.g., external sound system 106). For example, asdescribed above in connection with FIG. 1 , audio output 320 can includea 3.5 mm jack that accepts a 3.5 mm connector, a fiber optic audio port(e.g., that accepts a TOSLINK connector), an RCA jack, a High-DefinitionMultimedia Interface (HDMI) port, a Universal Serial Bus (USB) port(e.g., a USB 2.x port, a USB 3.x port, etc.), a THUNDERBOLT port, anyother suitable port and/or any suitable combination of ports. Speaker(s)321 can include any suitable number and/or type of speakers forpresenting audio and/or providing audio feedback to a user. In someimplementations, internal speaker(s) 321 can be omitted and/or computingdevice 102 can output audio to any other suitable device that can outputaudio. For example, in such implementations, computing device can outputaudio to another device to which it is connected by wire and/orwirelessly, such as a standalone speaker (e.g., connected by a speakerwire), a user device having an internal speaker (e.g., a smartphone, atablet computer, etc.), a computer having external speakers that isconnected to the same local area network, a wireless speaker to whichcomputing device 102 can connect (e.g., by Bluetooth connection), etc.

External sound system 106 can include a hardware processor 322, one ormore speakers 324, memory 326, and one or more audio inputs 328, whichcan be interconnected. In some implementations, memory 326 can include astorage device (such as a non-transitory computer-readable medium) forstoring a computer program for controlling hardware processor 322.

Hardware processor 322 can use the computer program to present audioreceived via audio input 328 and/or from any other source (e.g., from aninternal or external antenna that can be used to receive audio broadcastover the air).

Content delivery server 120 can include a hardware processor 332, adisplay/input device 334, memory 336, and a transmitter/receiver 338,which can be interconnected. In some implementations, memory 336 caninclude a storage device (such as a non-transitory computer-readablemedium) for storing a server program for controlling hardware processor332.

Hardware processor 332 can use the server program to communicate withuser device 112 and/or computing device 102, as well as provide accessto media content. It should also be noted that data received throughcommunications link 208 or any other communications links can bereceived from any suitable source. In some implementations, hardwareprocessor 332 can send and receive data through communications link 208or any other communication links using, for example, a transmitter,receiver, transmitter/receiver, transceiver, or any other suitablecommunication device, such as transmitter/receiver 338. In someimplementations, hardware processor 332 can receive commands and/orvalues transmitted by one or more users. Display/input device 334 caninclude a touchscreen, a flat panel display, a cathode ray tube display,a projector, a speaker(s), and/or any other suitable display and/orpresentation devices, and can further include a computer keyboard, acomputer mouse, a microphone, a touchpad, a voice recognition circuit, atouch interface of a touchscreen, and/or any other suitable inputdevice. Transmitter/receiver 318 can include any suitable transmitterand/or receiver for transmitting and/or receiving, among other things,media content including audio and/or video content.

In some implementations, content delivery server 120 can be implementedin one server or can be distributed as any suitable number of servers.For example, multiple servers 120 can be implemented in variouslocations to increase reliability and/or increase the speed at whichserver 120 can communicate with user devices 112 and/or computing device102.

In some implementations, content discovery server 122 can be implementedusing similar hardware to content delivery server 120, but can include aserver program for facilitating discovery and/or browsing of mediacontent items accessible from content delivery server 120.

FIG. 4 shows an example 400 of a process for controlling audio output inaccordance with some implementations of the disclosed subject matter. At402, process 400 can receive a request to present audio content. In someimplementations, the request to present audio content can be receivedfrom any suitable source and/or can be received using any suitabletechnique or combination of techniques. For example, in someimplementations, process 400 can receive the request to present audiocontent from a user device (e.g., user device 112), with identifyinginformation (e.g., a URL) that computing device 102 can use to requestthe audio content. As another example, in some implementations, process400 can receive the request to present audio content as spoken wordsthat are recorded by microphone 314 and converted into one or moreinstructions by hardware processor 312 (and/or any other suitableprocessor, such as a processor of a server with which computing device102 can communicate to interpret spoken user instructions). In such anexample, the request spoken by the user can be a request to presentparticular audio content (e.g., the name of an artist, an author, atitle of a song or book, etc.), a request to present content using aparticular application, a request to present a particular type ofcontent (e.g., a particular genre), any other suitable request forcontent, and/or any suitable combination thereof. As yet anotherexample, process 400 can receive a request to present content pushed tocomputing device 102 by another device (e.g., content delivery server120, user device 112, etc.). In such an example, the device pushing thecontent can send a request to computing device 102 to play the contentover any suitable communications link such as the Internet, a local areanetwork (e.g., Wi-Fi), a short range wireless connection (e.g.,Bluetooth), any other suitable communications link, or any suitablecombination of communications links.

At 404, process 400 can request and/or receive the audio content thatwas requested to be presented at 402. In some implementations, process400 can use any suitable technique or combination of techniques torequest and/or receive the audio content requested to be presented at402. For example, process 400 can request the content using an address(e.g., a URL) provided by user device 112. As another example, process400 can request the content by determining a search query and/orinstruction based on one or more words spoken by a user, and can use thesearch query and/or instruction to request content responsive to the oneor more words spoken by the user. As yet another example, process 400can receive audio content pushed by another device (e.g., user device112, content delivery server 120, etc.).

At 406, process 400 can determine whether an output to an external soundsystem (e.g., external sound system 106) is connected. In someimplementations, process 400 can use any suitable technique orcombination of techniques to determine whether an output to an externalsound system is connected. For example, process 400 can determinewhether an audio output port (e.g., connector 104) is connected to acorresponding plug. In a more particular example, process 400 candetermine whether a 3.5 mm plug is inserted into the 3.5 mm jack. Insuch examples, process 400 can use any suitable technique or combinationof techniques to determine that a plug is inserted into an output port,such as determining whether a metal contact of the plug is present inthe output port. In another more particular example, process 400 candetermine whether a particular signal is received at a contact of theoutput port, such as a hot plug detect signal that is output on pin 18of an HDMI port and received back at pin 19 of the HDMI port when anHDMI sink is connected to the HDMI port (e.g., via an HDMI cable). Asanother example, computing device 102 can determine whether audiocontent is being output over a wireless connection that is used totransmit audio content to external sound system 106.

If process 400 determines that there is no output to external soundsystem 106 connected (“NO” at 406), process 400 can proceed to 414, andoutput audio using one or more internal speakers (e.g., speaker(s) 321).Otherwise, if process 400 determines that an output to external soundsystem 106 is connected (“YES” at 406), process 400 can proceed to 408.

At 408, process 400 can output the audio received at 404 over theconnection to external sound system 106 that is connected. In someimplementations, process 400 can inhibit the audio being output over theconnection to external sound system 106 from also being presented usingspeaker(s) 321. Additionally, in some implementations, process 400 canoutput the audio over only a single output port in cases in whichmultiple output ports are present and connected to external soundsystems (which may be the same external sound system 106 or differentexternal sound systems). Alternatively, in some implementations, process400 can output the audio over any output ports for which a connection isdetected. In such implementations, the audio output over differentoutput ports can be output with different timing in situations in whichdifferent external sound systems exhibit different delays.

At 410, process 400 can record one or more audio samples usingmicrophone 314, which can be used to determine whether the audio beingoutput at 408 is being presented by an external speakers (e.g.,speaker(s) 324 of external sound system 106). In some implementations,process 400 can record any suitable amount of audio and/or any suitablefrequencies of audio for comparison with the audio output at 408. Forexample, process 400 can cause the audio to be continuously recorded,and can use a particular portion of the recorded audio to compare to theaudio output at 408 (e.g., as described below in connection with 412).As another example, process 400 can record samples at discrete timeintervals, without recording signals from sounds that occur between thetime intervals (e.g., process 400 can capture samples every quartersecond, every fifth of a second, etc.). As yet another example, process400 can record audio at particular frequencies (e.g., by filtering outor otherwise discarding audio at other frequencies).

In some implementations, signals recorded at 410 and signals recorded inorder to determine whether the user has issued voice commands aredeleted, overwritten and/or otherwise disposed of when computing device102 has used the signal (e.g., to determine whether audio is beingplayed by an external speaker, whether the user has issued a voicecommand, etc.). Additionally, in some implementations, microphone 314can be disabled when not in use and/or and signals output by microphone314 can be disregarded (e.g., never recorded in memory 316) when theuser has not requested that the signals be recorded. For example, insome implementations, signals from microphone 314 can be recorded (e.g.,in memory 316) when audio content is being output using audio output 320in order to determine whether external sound system 106 is presentingthe audio content. As another example, signals from microphone 314 canbe recorded (e.g., in memory 316) when the user is giving a voicecommand in order to determine one or more instructions based on thevoice command. In such an example, a user can be required to say aparticular word or combination of words (e.g., a trigger word or triggerphrase) and/or perform some other recognized action, such as pressing aphysical button to initiate recording of subsequent words that are toconstitute a voice command. Until the trigger word or trigger phrase isrecognized, signals from microphone 314 can be discarded.

At 412, process 400 can determine whether the audio recorded at 410matches the audio output at 408. Process 400 can use any suitabletechnique or combination of technique to determine whether the recordedaudio matches the outputted audio. For example, in some implementations,process 400 (and/or any other suitable process) can generate one or moreaudio fingerprints based on the recorded audio, and compare thegenerated fingerprints to one or more audio fingerprints of the audiobeing output at 408. In such an example, the fingerprints of therecorded audio and/or the fingerprints of the outputted audio can begenerated by computing device 102 and/or can be generated by a remotecomputer (e.g., a remote server). In some implementations, thefingerprints of the outputted audio can be generated as the audio isoutput. Additionally or alternatively, the fingerprints of the outputtedaudio can be generated prior to the content being output (e.g., in adatabase of known audio content), and the fingerprints of the recordedaudio can be compared to fingerprints from the previously generatedfingerprints.

As another example, in some implementations, process 400 (and/or anyother suitable process) can compare the recorded audio signal to theoutputted audio signal to determine whether the signals match. In a moreparticular example, process 400 can take a difference between therecorded audio signal and the outputted audio signal. In such a moreparticular example, if the result of the difference is below athreshold, process 400 can determine that the outputted signal and therecorded signal match. In some implementations, delays caused bytransmitting the audio signal to external sound system 106, delayscaused by external sound system 106 and/or delays introduced whenrecording the audio content can cause a comparison between the audiocurrently being output and the recorded audio to be different regardlessof whether the audio is being presented by external sound system 106. Insome implementations, process 400 can compare the recorded audio to atime-delayed version of the signal that was output to external soundsystem 106. Process 400 can use any suitable technique or combination oftechniques to determine the amount of delay, such as by matching apattern of local maxima from both signals to attempt to match the timingof the signals, by comparing the recorded signal to versions of theoutput signal with multiple different amounts of delay, by determiningthe delay between the output audio and the recorded audio once (e.g.,using a calibration process) and using the determined delay, etc. Insome implementations, process 400 (and/or any other suitable process)can compensate for background noise captured in the recording using anysuitable technique or combination of techniques, such as by filteringthe audio signal based on the known signal (e.g., the output audiosignal). Note that in most cases the recorded audio will not match theoutput audio exactly (e.g., due to the hardware being used to play theaudio, the hardware being used to record the audio, ambient noise, theacoustics or the room, etc.), and accordingly, process 400 can determinewhether the recorded audio is substantially similar to the output audiosignal. For example, process 400 (and/or any other suitable process) candetermine a confidence value that represents how closely the recordedaudio matches the output signal. In such an example, if the confidencevalue is greater than a threshold value, process 400 can determine thatthe recorded audio matches the audio being output.

In some implementations, process 400 can alter the audio signal outputat 408 to generate a modified audio signal. In such implementations,process 400 can introduce one or more signals or alterations to theexisting signal to increase the likelihood that the recorded audiosignal can be recognized without substantially changing how the outputaudio signal sounds to most human listeners. For example, process 400can add a signal having a particular pattern at a frequency that can beoutput by most external speakers, but that is outside of the range ofhuman hearing. As another example, process 400 can change the amplitudeof the signal in a particular pattern such that it is unlikely to beobserved by a listener, but is recognizable by process 400 (and/or anyother suitable process). In such implementations, process 400 candetermine whether the pattern of the inserted signal and/or the changein amplitude is present in the recorded signal to determine whether theoutputted audio signal is being presented by external audio system 106.

If process 400 determines that the recorded audio matches the outputtedaudio signal (“YES” at 412), process 400 can return to 406 and cancontinue to output the audio if the output to the external sound systemis connected.

Otherwise, if process 400 determines that the recorded audio does notmatch the outputted audio signal (“NO” at 412), process 400 can proceedto 414. At 414, process 400 can output the audio signal using speaker(s)321, and can return to 406 to determine whether an output to an externalaudio system is connected. The recorded audio may not match theoutputted audio signal for various reasons, such as the external soundsystem being powered down, a different input of the external soundsystem being selected, the output not being connected to the externalaudio system, a malfunction, etc.

In some implementations, during periods when process 400 is presentingaudio content using speaker(s) 321 at 414, process 400 can provide audioto audio output 320 such that, if and when external sound system 106becomes configured to present audio content, process 400 can provide theaudio content at 408 and inhibit presentation of the audio content byspeaker(s) 321 at 414. In some implementations, process 400 can provideany suitable audio to audio output 320 at 414. For example, an audiosignal outside the range of human hearing can be provided, and process400 can determine, at 412, whether that signal is present in audiorecorded at 410. As another example, the audio content received at 404can be provided. In such an example, in cases in which the audio contentis provided to audio output 320, the audio content may be presented byboth the internal speaker and the external speakers (e.g., speaker(s)324 of external sound system 106) with a delay between the two, and bothmay be present in audio recorded at 410. In some implementations,process 400 (and/or any other suitable process) can use any suitabletechnique or combination of techniques to distinguish between the audiopresented by speaker(s) 321 and speaker(s) 324. For example, in someimplementations, process 400 can subtract the signal provided tospeaker(s) 321 from the recorded signal based on a known timingrelationship between when audio is presented by speaker(s) 321 and whenthat same audio is recorded by microphone 314. As another example,multiple microphones 314 can record sound, and can be placed in relationto speaker(s) 321 such that the signals from the multiple microphonescan be modified and/or combined (e.g., by subtracting the output of onemicrophone from the output of another, by inverting the output from onemicrophone and adding it to another, etc.) such that the signal providedto speaker(s) 321 is removed. In such an example, the residual signalcan be used to determine whether the audio content is present in therecorded signal as presented by external sound system 106.

In some implementations, process 400 can be used for some audio to bepresented and not for other audio to be presented. For example, in someimplementations, audio that is output to interact with a user (e.g.,prompts, queries, responses to queries, etc.) can always be presentedusing only speaker(s) 321.

In some implementations, the mechanisms described herein, can includeserver-side software, server-side hardware, client-side software,client-side hardware, or any suitable combination thereof. For example,the mechanisms can encompass a computer program written in a programminglanguage recognizable by hardware processor 312, hardware processor 322,and/or hardware processor 332 (e.g., a program written in a programminglanguage, such as, Java, C, Objective-C, C++, C#, Javascript, VisualBasic, or any other suitable approaches). As another example, theapplications can encompass code corresponding to one or more Web pagesor Web page portions (e.g., via any suitable encoding, such as HyperText Markup Language (“HTML”), Dynamic Hyper Text Markup Language(“DHTML”), Extensible Markup Language (“XML”), JavaServer Pages (“JSP”),Active Server Pages (“ASP”), Cold Fusion, or any other suitableapproaches).

In some implementations, any suitable computer readable media can beused for storing instructions for performing the functions and/orprocesses described herein. For example, in some implementations,computer readable media can be transitory or non-transitory. Forexample, non-transitory computer readable media can include media suchas magnetic media (such as hard disks, floppy disks, etc.), opticalmedia (such as compact discs, digital video discs, Blu-ray discs, etc.),semiconductor media (such as flash memory, electrically programmableread only memory (EPROM), electrically erasable programmable read onlymemory (EEPROM), etc.), any suitable media that is not fleeting ordevoid of any semblance of permanence during transmission, and/or anysuitable tangible media. As another example, transitory computerreadable media can include signals on networks, in wires, conductors,optical fibers, circuits, any suitable media that is fleeting and devoidof any semblance of permanence during transmission, and/or any suitableintangible media.

It should be understood that the above described steps of the processesof FIG. 4 can be executed or performed in any order or sequence notlimited to the order and sequence shown and described in the figures.Also, some of the above steps of the processes of FIG. 4 can be executedor performed substantially simultaneously where appropriate or inparallel to reduce latency and processing times.

It should also be noted that, as used herein, the term mechanism canencompass hardware, software, firmware, or any suitable combinationthereof.

Accordingly, systems, methods, and media for controlling audio outputare provided.

Although the invention has been described and illustrated in theforegoing illustrative implementations, it is understood that thepresent disclosure has been made only by way of example, and thatnumerous changes in the details of implementation of the invention canbe made without departing from the spirit and scope of the invention,which is limited only by the claims that follow. Features of thedisclosed implementations can be combined and rearranged in variousways.

What is claimed is:
 1. A method for controlling audio output, the methodcomprising: determining, using an audio input device of a computingdevice to captures an audio clip portion being played back by aconnected audio device, whether audio content that has been transmittedto the connected audio device is being played back by the connectedaudio device; and inhibiting, using a hardware processor of thecomputing device, the audio content from being transmitted to theconnected audio device and automatically switching the audio content tobe presented using an audio output device of the computing device basedon the determination that the audio content that has been transmitted tothe connected audio device is not being played back by the connectedaudio device.